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PREFACE 


This  paper  is  an  introduction  to  the  basic  methodologies  and 
modeling  concepts  which  underlie  the  Training  Decisions  System 
(TDS)  and  the  Training  Decisions  Modeling  Technologies  (TDMT) 
currently  under  development  at  the  Armstrong  Laboratory.  The 
reader  is  encouraged  to  seek  out  more  specific  information  on 
these  technologies  by  accessing  the  numerous  publications  and 
presentations  cited  in  this  paper  and  its  appendices. 
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SUMMARY 


Effective  training  planning  and  programming  is  crucial  to 
the  operation  of  an  organization.  Unfortunately,  the  environment 
in  which  training  decisions  are  being  made  in  the  United  States 
Air  Force  is  characterized  by  a  decentralized  management  system 
and  a  lack  of  readily  available  data  upon  which  to  base 
decisions.  Training  planners  are  consequently  forced  to  make 
training  decisions  without  reliable  estimates  of  the  cost  and 
capacity  implications  of  their  decisions.  In  an  era  of  fiscal 
constraint,  it  is  imperative  that  training  planners  be  provided 
with  this  information.  The  AF  has  undertaken  the  Training 
Decisions  Modeling  Technologies  research  program  to  address  this 
problem.  The  Training  Decisions  System  (TDS),  which  forms  the 
baseline  technologies  for  the  research  program,  uses  information 
about  jobs  performed  by  airmen,  personnel  assignment  flows, 
course  training  content,  and  training  resources  to  determine 
training  capacities  and  the  most  cost-effective  training  options 
available.  TDS  develops  a  model  of  a  specialty's  Utilization 
and  Training  (U&T)  Pattern  according  to  data  collection  results 
and  simulates  the  flow  of  airmen  through  it.  Based  on  this 
simulation,  TDS  tabulates  the  costs  associated  with  training  and 
evaluates  the  training  capacity  of  the  training  system.  Specific 
technologies  developed  within  TDS,  such  as  Task  Module 
construction  and  allocation  curves,  have  greatly  enhanced  our 
training  decisions  modeling  capabilities. 


Introduction  to  Training  Decisions  Modeling  Technologies:  The 

Training  Decisions  System 


Training  planning  and  programming  is  a  vital  part  of  any 
training  system,  and  an  effective  training  system  is  paramount  to 
the  successful  operation  of  an  organization.  Yet,  policy  makers 
and  training  managers  are  being  asked  to  make  decisions  about 
training  —  decisions  which  will  affect  the  entire  organization 
—  without  the  necessary  information  or  tools  they  require. 

Policy  makers  and  training  managers  operate  in  an  extremely 
complex  environment.  They  must  often  make  decisions  based  on 
limited  information  and  best  guesses  about  what  to  train,  where 
to  train,  and  when  in  a  worker's  career  to  provide  training. 
Moreover,  these  training  planners  are  unaware  of  many  of  the 
potential  impacts  or  consequences  of  their  decisions. 

This  paper  presents  an  overview  of  the  Training  Decisions 
Modeling  Technologies  currently  under  development  by  the  United 
States  Air  Force  to  facilitate  and  enhance  the  quality  of 
training  planning  decisions.  We  will  present  a  sample  training 
problem  and  examine  the  process  by  which  the  Training  Decisions 
System  (TDS)  —  which  forms  the  baseline  technologies,  methods, 
and  models  of  this  research  program  —  collects,  processes,  and 
provides  outputs  of  relevant  cost  and  capacity  data  in  order  to 
facilitate  AF  training  planning  decision  making.  Detailed 
information  is  available  on  this  line  of  research  in  a  number  of 
separate  publications.  A  list  of  publications  and  presentations 
associated  with  TDMT  research  program  can  be  found  in  Appendices 
A  and  B,  respectively. 

The  US  Air  Force  Training  System 

Training  in  the  AF  is  an  essential  and  integral  part  of 
every  airman's  career.  Training  begins  with  basic  military 
training  and  continues  throughout  the  remainder  of  an  airman's 
career.  Each  year,  thousands  of  airmen  are  provided  technical 
training  in  over  200  AF  specialties  (i.e.,  occupational 
categories).  The  AF  training  system  includes  formal  courses  for 
resident  students  conducted  at  Technical  Training  Centers  (TTCs); 
continuing  on-the-job  training  (OJT)  performed  at  field  units; 
and  additional  training  accomplished  in  a  number  of  other 
settings,  including  ATC  Field  Training  Detachments  (FTDs),  Career 
Development  Courses  (CDCs),  Major  Command  (MAJCOM)  programs, 
mobile  training  teams,  contractor-provided  training,  and 
inter-service  programs  and  courses. 

Technical  training  within  the  AF,  including  initial 
technical  training  and  follow-on  training,  is  an  expensive 
undertaking.  Current  estimates  place  the  annual  technical 
training  budget  at  approximately  one  billion  dollars.  AF 
training  planners  are  tasked  with  the  large  responsibility  of 
managing  the  AF  training  system  and  developing  cost-effective 
training  programs  for  each  AF  specialty.  The  size  and  complexity 
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of  the  AF  training  system,  however,  creates  considerable  problems 
for  AF  training  planners  and  constrains  their  ability  to  make 
decisions  about  the  most  effective  and  efficient  way  to  allocate 
job  training  content  across  alternative  training  settings — what 
tasks  should  be  trained  in  which  settings  (e.g.,  technical 
training  centers,  on-the-job  training) — and  at  what  points  in  an 
airman's  career  certain  training  content  would  be  most 
beneficial.  A  decentralized  training  management  system,  limited 
training  resources,  and  the  lack  of  readily  available  information 
upon  which  to  base  cost  comparisons  are  among  the  factors 
complicating  the  decision  making  process. 

The  bottom  line  is  that  training  planners,  such  as  the 
functional  managers  of  Air  Force  specialties,  must  make  training 
decisions  without  reliable  estimates  of  their  cost  and  capacity 
implications.  Many  of  the  costs  associated  with  training  simply 
elude  AF  training  planners  because  of  the  dearth  of  available 
information  on  training  content,  training  demand,  the  number  of 
personnel  moving  through  jobs  and  training,  resources  required 
for  training,  and  the  number  of  resources  available  to  provide 
training.  Some  costs,  like  those  associated  with  on-the-job 
training,  are  hidden  costs  which  have  traditionally  been 
neglected.  This  is  in  part  due  to  the  absence  of  OJT  cost 
accounting  procedures  and  the  fact  that  these  costs  are  subsumed 
under  the  operational  budget.  Training  capacity  information  is 
also  scarce.  The  training  system  can  only  process  a  certain 
number  of  airmen  given  its  resource  constraints.  Unfortunately, 
there  is  no  systematic  means  by  which  these  constraints  can  be 
evaluated.  In  an  era  of  fiscal  constraint,  it  is  imperative  that 
training  decisions  be  made  with  full  knowledge  of  the  associated 
cost  and  capacity  implications. 

The  United  States  Air  Force  has  recognized  the  unwieldy 
problems  which  face  those  persons  responsible  for  making  AF 
training  decisions  and  has  undertaken  a  research  program  to 
address  them.  The  present  approach  to  these  problems  has  been 
the  development  of  a  computer-based  decision  modeling  technology 
known  as  the  Training  Decisions  System  (TDS).  The  baseline 
technologies  developed  under  this  research  program  integrate 
training  requirements,  as  well  as  manpower  and  cost 
considerations,  into  a  single  comprehensive  model.  These 
technologies  use  information  about  jobs  performed  by  airmen, 
personnel  assignment  flows,  course  training  content,  and  training 
resources  to  determine  training  capacities  and  the  most 
cost-effective  training  options  available.  The  present  version 
of  the  TDS  consists  of  three  basic  subsystems  and  a  fourth 
integrating  subsystem;  the  Task  Characteristic  Subsystem,  the 
Field  Utilization  Subsystem,  the  Resource  Cost  Subsystem,  and  the 
Integration  and  Optimization  Subsystem. 

Sample  AF  Problem 

A  meeting  of  senior  level  AF  officers  (colonels)  was 
recently  convened  in  order  to  review  AF  maintenance  training. 
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This  group  of  officers,  known  as  the  Maintenance  Training 
Advisory  Group  (MATAG) ,  identified  a  need  to  determine  the  impact 
of  reduced  TDY-to-school  funding  (training  which  occurs  at  a 
location  away  from  an  airman's  workplace)  on  on-the-job  training 
(OJT)  and  the  AF  mission.  Budget  cutbacks  have  not  yet  had  an 
impact  on  TDY-to-school  funding,  but  the  group  noted  that  it  was 
only  a  matter  of  time  before  the  cutbacks  do  have  an  effect. 

The  advisory  group's  main  concern  was  to  identify  those  AF 
specialties  which  were  the  best  candidates  for  reduced 
TDY-to-school  funding  given  the  potential  for  cutbacks  in  this 
area.  This  sort  of  inquiry  reinforces  our  earlier  observation 
that  training  planners  are  inadequately  prepared  to  make  training 
planning  decisions  at  the  present  time.  For  reasons  previously 
noted,  training  planners  are  unable  to  determine  the  impact  of 
reduced  TDY-to-school  funding  on  OJT  costs,  let  alone  the  impact 
on  costs  associated  with  the  entire  training  system.  Moreover, 
training  planners  are  unable  to  determine  whether  or  not  the 
training  system  can  logistically  support  alternative  training 
programs;  i.e.,  operational  units  may  not  have  the  necessary 
labor  and/or  non-labor  resources  to  support  an  increase  in  OJT  if 
the  training  burden  should  shift  away  from  formal  courses. 

The  Training  Decisions  Modeling  Technologies  currently  under 
development  by  the  AF  can  aid  training  planners  in  making  their 
decisions  by  providing  them  with  macro-level  cost  and  capacity 
information  about  the  AF  training  system.  In  order  to  illustrate 
this  point  and  demonstrate  the  capabilities  of  the  current 
Training  Decisions  Modeling  Technologies,  we  will  examine  how  TDS 
can  help  inform  training  planning  about  a  real  world  question  — 
i.e.,  what  is  the  impact  of  reduced  TDY-to-school  funding  on  OJT 
—  for  the  Aircraft  Environmental  Systems  (423X1)  specialty. 

Training  Decisions  System  (TDS) 

Decision  Factors.  Training  planners  for  the  Aircraft 
Environmental  Systems  job  specialty  must  determine  what  the 
impact  will  be  on  their  OJT  requirements  if  the  opportunity  for 
formal  training  is  reduced  due  to  budget  cutbacks.  If  we 
conceptualize  such  a  problem,  we  can  see  that  many  outcomes  are 
possible.  For  example,  a  reduction  in  TDY-to-school  funding 
could  adversely  affect  overall  training  costs  for  the  423X1 
specialty  by  dramatically  increasing  OJT  requirements  and  costs. 
On  the  other  hand,  OJT  requirements  and  costs  may  be  only 
slightly  increased  and  therefore  result  in  an  overall  savings. 

The  primary  consideration  here  is  cost,  but  other  factors 
must  be  considered  along  with  cost  in  order  to  make  a  fair 
assessment  of  the  impact  of  reduced  TDY-to-school  funding. 
Training  planners  also  require  information  about  the  effects  of 
reduced  funding  on  training  capacities  —  the  number  of  airmen 
the  training  system  can  process.  Alternative  training  programs 
are  not  viable  if  they  cannot  support  a  specialty's  training 
requirements.  Training  proficiency  is  a  third  factor  deserving 
of  consideration.  However,  since  TDS  purposely  models  airmen 
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such  that  they  become  fully  proficient  on  the  tasks  being 
trained,  training  decision  impacts  on  training  and  job 
performance  are  not  modeled.  Future  research  will  be  directed  at 
modeling  the  impact  of  training  decisions  on  these  issues. 

TPS  Overview.  In  the  above  discussion,  we  determined  that 
training  planners  require  at  least  two  fundamental  types  of 
information  (assuming  airmen  must  be  trained  to  full  proficiency) 
to  make  an  informed  decision  about  the  impacts  of  reduced 
TDY-to-school  funding.  That  is,  they  require  information  about 
the  cost  and  capacity  implications  of  their  decisions.  We  will 
now  examine  the  process  by  which  training  planners  go  about 
implementing  TDS  in  order  to  obtain  the  necessary  information 
about  their  job  specialty. 

TDS  essentially  operates  by  developing  a  dynamic  model  of  an 
AF  specialty's  utilization  and  training  (U&T)  pattern  —  i.e., 
the  typical  flow  of  airmen  through  the  job  and  training  states 
associated  with  a  given  specialty  (Figure  1); 


Figure  1 .  Generic  Utilization  and  Training  Pattern 
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Figure  1  represents  the  flow  of  airmen  from  their  initial 
training  (e.g.  Basic  Training)  to  their  subsequent  job 
assignments.  During  each  job  assignment  or  phase,  airmen  receive 
both  formal  course  and  on-the-job  training.  This  training  is 
mandated  by  either  the  particular  job  an  airman  performs  or  by  an 
airman's  time  in  service.  Using  this  model  as  a  baseline,  TDS 
simulates  the  flow  of  airmen  through  the  U&T  pattern  and 
determines  the  impacts  of  training  decisions,  such  as  reduced 
TDY-to-school  funding,  on  the  costs  and  capacities  of  the 
training  system.  Consequently,  collecting  the  necessary  data  to 
describe  and  model  a  specialty's  U&T  pattern  is  the  first  step  in 
a  TDS  analysis  (see  Mitchell,  Vaughan,  Yadrick,  Collins,  and 
Hernandez,  1988;  Yadrick,  Knight,  Mitchell,  Vaughan,  and  Perrin, 
1988)  . 

The  objective  of  the  data  collection  effort  and  its 
associated  procedures  is  to  develop  an  accurate  picture  of  the  AF 
training  system  as  it  pertains  to  a  particular  AF  specialty. 

This  is  accomplished  in  several  steps: 

1.  Jobs  and  training  courses  for  a  particular  AF  specialty 
must  be  identified. 

2.  Next,  the  jobs  and  training  courses  which  are  identified 
for  the  specialty  must  be  described  in  terms  of  their  task 
composition.  The  TDS  uses  Task  Modules  (TMs)  —  i.e.,  groups 
of  tasks  sharing  similar  characteristics  —  for  this 
purpose . 

3.  Training  course  and  job  assignment  flows  must  also  be 
identified  in  order  to  build  a  U&T  pattern  for  the  specialty 
(figure  1,  above).  This  pattern  will  be  used  as  a  baseline 
for  simulating  the  impact  of  training  decisions. 

4 .  Resource  and  cost  information  is  developed  for  each  task 
module  (TM)  in  a  given  training  setting  (i.e.,  technical 
training  center,  field  training  detachment,  on-the-job 
training,  correspondence,  etc...).  This  allows  TDS  to 
describe  each  of  the  training  courses  within  a  U&T  pattern 
in  terms  of  cost  and  resource  requirements. 

5.  Finally,  information  for  describing  and  developing 
representative  sites  —  hypothetical  AF  units  used  to 
characterize  several  similar  actual  AF  units  --  must  be 
collected  for  the  specialty.  These  sites  are  used  to 
calculate  TDY  and  travel  costs  and  show  variations  in  the 
availability  of  resources  for  training  purposes.  Based  on 
this  information,  the  TDS  model  can  determine  the  capacity 
limitations  of  the  training  system--i . e . ,  the  correspondence 
or  disparity  between  resources  available  for  training  and 
resources  required  for  training. 

The  information  collected  in  the  above  steps  is  used  to 
develop  a  model  of  the  current  U&T  pattern  for  a  particular  AF 
specialty.  TDS  simulates  the  flow  of  a  specified  number  of  airmen 
through  the  U&T  pattern  based  on  the  specific  transition 
probabilities  (to  jobs  and  to  training  courses)  identified  for 
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the  pattern.  Each  simulated  airman  receives  training  in  both 
formal  (courses)  and  informal  settings  (OJT)  based  on  the  set  of 
TMs  associated  with  their  specific  jobs.  Training  costs  are 
calculated  for  each  airmen  and  aggregated  for  the  entire 
specialty. 

Utilization  and  Training  <U&T)  Patterns.  U&T  patterns  are 
composed  of  two  basic  elements:  jobs  and  training  courses.  The 
initial  tasking  of  a  TDS  analyst  is  to  identify  the  training 
course  and  job  assignment  flows  which  characterize  a  given 
specialty  and  describe  each  job  and  training  course  within  the 
specialty  in  terms  of  its  content  (task  composition).  At  present, 
there  is  no  central  data  base  containing  all  the  necessary 
information  about  jobs  and  courses.  This  information  must  be 
collected  through  a  combination  of  data  collection  techniques 
including  surveys,  interviews  with  subject  matter  experts  (SMEs), 
and  reviews  of  various  records  and  data  bases  (references). 

An  analysis  of  the  Aircraft  Environmental  Systems  specialty 
reveals  that  it's  U&T  pattern  is  composed  of  essentially  ten 
different  jobs  and  their  associated  training  courses  (Figure  2). 
The  jobs  identified  in  this  diagram  do  not  necessarily  correspond 
to  those  identified  in  other  AF  data  bases.  TDS  classifies  jobs 
according  to  their  task  composition;  other  AF  data  bases  may  have 
different  classification  schemes  (Shartle,  1959).  For  a  more 
complete  account  of  job  identification  within  TDS  see  Yadrick  et 
al.,  1988. 

At  this  point  in  the  TDS  analysis,  the  U&T  pattern,  like  the 
one  developed  for  the  423X1  specialty,  is  merely  a  shell  which 
must  be  filled  in.  Each  job  and  training  course  within  the 
initial  framework  requires  further  elaboration.  Jobs  must  be 
described  in  terms  of  their  tasks;  that  is,  in  terms  the  tasks 
which  a  person  within  that  job  is  expected  to  perform.  Training 
courses  must  also  be  described  in  terms  of  the  tasks  they  train. 
As  will  be  seen,  the  TDS  uses  the  information  collected  about  job 
and  training  course  content  to  determine  the  extent  to  which  each 
task  is  trained  at  the  formal  schools  and/or  on-the-job. 

Job  and  Course  Descriptions:  Task  Module  Construction.  TDS 
describes  both  job  and  training  content  in  terms  of  groupings  of 
tasks  called  task  modules  (TMs),  rather  than  individual  task 
statements.  The  use  of  task  level  data  is  widespread  in  the  Air 
Force,  but  TMs  were  chosen  for  use  in  the  TDS  because  they  avoid 
a  number  of  problems  inherent  in  task  level  data  (see  Perrin, 
Knight,  Mitchell,  Vaughan,  &  Yadrick,  1988).  TMs  serve  as  the 
basic  units  of  analysis  in  the  TDS  and  each  of  its  subsystems. 
They  are  used  to  describe  job  and  training  content  and  are  the 
basis  for  determining  costs  and  resource  requirements. 

Normal  task  analysis  procedures  often  result  in  repetitive 
data  because  many  tasks  share  common  skills  and  knowledges.  TMs 
avoid  this  problem  by  grouping  together  tasks  with  similar  skills 
and  knowledges.  The  resulting  TMs  increase  the  efficiency  of  the 
training  analysis  process  and  reduce  the  possibility  of 
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Figure  2 .  Sample  UiT  Pattern  for  AFS  423X1  (Mitchell  & 
Yadrick,  1989) 

overestimating  training  requirements.  TMs  are  also  more 
manageable  than  task  level  data.  Since  each  AF  specialty  has 
between  300  to  2000  tasks  associated  with  it,  the  use  of  task 
level  data  can  be  very  taxing,  not  to  mention  time  consuming  and 
expensive.  TMs  organize  tasks  into  coherent,  manageable  bundles, 
(see  also  Perrin  &  Bennett,  1989) 

Procedures  for  TM  construction  are  found  in  the  Task 
Characteristic  Subsystem  of  the  TDS.  These  procedures  were 
arrived  at  after  an  empirical  evaluation  of  two  separate 
approaches  during  the  initial  research  and  development  phase  of 
TDS  (Perrin  et  al.,  1988).  The  first  to  be  evaluated  was  a 
judgmental  approach.  In  this  approach,  subject  matter  experts 
(SMEs)  were  asked  to  group  together  those  tasks  within  their 
specialty  which  they  believed  should  be  trained  together.  It  was 
assumed  that  consensus  judgements  of  several  of  SMEs  about  which 
tasks  within  their  specialty  should  be  trained  together  were  the 
best  criterion  against  which  to  judge  TM  construction.  The 
judgmental  approach,  however,  had  some  drawbacks.  First, 
disparity  existed  between  the  TMs  formed  by  separate  SMEs.  SMEs 
tended  to  form  TMs  according  to  their  own  experiences  and 
perspectives.  It  was  only  after  extended  negotiations  that  a 
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fair  —  but  not  complete  —  consensus  was  obtained.  A  second 
major  drawback  to  this  approach  was  the  large  amount  of  time  it 
consumed. 

The  second  approach  to  be  evaluated  involved  multivariate 
statistical  clustering  (Perrin  &  Bennett,  1989).  Multivariate 
techniques  were  used  in  an  iterative  manner  with  a  variety  of 
input  data  including  the  probability  of  task  co-performance 
(derived  from  occupational  survey  data),  common  equipment  usage, 
and  skill-level  information.  It  was  found  that  the  latter  two 
variables  added  little,  if  any,  refinement  to  those  clusters 
formed  on  the  basis  of  co-performance.  The  Comprehensive 
Occupational  Data  Analysis  Program  [CODAP]  software  system  is 
currently  the  primary  tool  for  advanced  work  on  the  hierarchical 
clustering  of  tasks. 

Co-performance  clustering  represented  a  considerable  time 
savings  when  compared  to  the  judgmental  approach  and  provided  a 
fair  approximation  of  SME  judgements.  The  recommended  approach 
for  task  module  construction  was  a  combination  of  statistical 
clustering  and  SME  refinement  (Perrin  et  al.,  1988). 
Co-performance  clusters  were  formed  and  subsequently  refined  by 
SMEs.  SMEs  provided  a  descriptive  title  for  each  TM,  placed 
single  unclustered  tasks  in  the  appropriate  TMs,  and  ensured  the 
overall  quality  of  the  TMs.  Task  analysis  has  revealed  that  TMs 
constructed  under  these  procedures  contained  tasks  with  similar 
skills  and  knowledge  requirements. 

TMs  are  used  to  describe  the  content  of  each  job  and 
training  course  identified  in  the  U&T  pattern.  Each  job  has  a 
unique  set  of  TMs  associated  with  it.  Similarly,  each  training 
course  provides  training  on  a  unique  set  of  TMs.  (TDS  describes 
training  courses  as  though  they  are  ideal  courses  —  the  TMs 
identified  for  a  particular  course  and  actual  training  course 
content  don't  always  coincide  exactly.) 

Resource  and  Cost  Information.  A  major  objective  of  TDS  is 
to  provide  training  planners  with  cost  and  resource  information 
about  their  specialty.  In  order  to  do  this,  TDS  estimates  the 
quantity  of  1)  instructor  hours,  2)  student  hours,  and  3) 
non-labor  resource  hours  devoted  to  training  each  TM  within  a 
given  training  setting.  These  data  are  collected  by  means  of  a 
questionnaire  or,  in  the  case  of  student  hours,  based  on  the 
simulated  flow  of  airmen  through  the  U&T  pattern.  Estimated 
instructor  hours  and  non-labor  resource  hours  are  used  to 
determine  resource  requirements  for  each  TM.  As  will  be  seen 
later,  TDS  compares  these  resource  requirements  to  resource 
availability  to  determine  whether  or  not  training  capacities  are 
exceeded.  Readily  available  information  about  non-labor 
resources,  such  as  the  types  and  amounts  of  resources  required  to 
train  a  particular  TM  in  a  given  training  setting,  is 
surprisingly  scarce  due  to  a  decentralized,  unstandardized 
training  management  system. 

Labor  costs  are  based  on  the  hourly  salaries  of  students  and 
instructors  and  the  estimated  quantities  of  student  and 
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instructor  hours  for  each  TM.  These  costs  are  easily  calculated 
for  formal  settings  where  information  regarding  training  times  is 
readily  obtainable  through  surveys.  Procedures  for  determining 
the  labor  costs  in  informal  or  OJT  settings,  however,  are  a  bit 
more  involved. 

On-the-Job  Training  Requirements;  Task  Module  Allocation 
Curves .  Determining  OJT  requirements  has  historically  been  a 
difficult  task.  Every  unit  within  the  AF  seems  to  conduct  OJT  in 
their  own  peculiar  way.  It  is  virtually  impossible  to  ascertain 
the  amount  of  time  spent  on  OJT  in  general  and,  more  precisely, 
on  training  specific  tasks.  Consequently,  the  labor  costs 
associated  with  OJT  have  remained  hidden.  TDS  overcomes  this 
problem  by  means  of  allocation  curves.  These  curves  (figure  3) 
show  the  relationship  between  training  time  and  proficiency  for  a 
given  TM  in  different  training  settings. 

In  Figure  3,  we  see  allocation  curves  for  four  different 
training  settings:  on-the-job  training,  technical  training 
centers,  field  training  detachments,  and  correspondence.  These 
curves  are  derived  from  SME  judgements  of  the  current,  ideal,  and 
maximum  training  times  for  each  TM.  (see  Perrin  et  al.,  1988; 
Vaughan,  Mitchell,  Yadrick,  Perrin,  Knight,  Eschenbrenner , 

Rueter,  &  Feldsott,  1989)  The  curves  are,  in  most  cases, 
negatively  accelerating.  This  plateauing  of  the  curves 
represents  the  maximum  proficiency  attainable  for  a  particular  TM 
in  a  given  training  setting.  Combinations  of  training  settings 
and  training  times  can  be  used  to  reach  full  proficiency.  In 
other  words,  proficiencies  for  a  given  TM  are  additive  across 
training  settings. 


Hours  of  Training 

Figure  3.  Seunple  Task  Module  Allocation  Curve 
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TDS  uses  allocation  curves  to  determine  the  number  of 
training  hours  beyond  those  already  expended  in  formal  courses 
necessary  to  bring  an  airman  up  to  full  proficiency  on  a  given 
TM.  Using  information  about  the  amount  of  time  devoted  to  a 
particular  TM  in  formal  courses,  TDS  can  determine  the  amount  of 
proficiency  an  airman  has  attained  on  that  TM.  Assuming  all 
airmen  must  be  trained  to  full  proficiency,  TDS  can  calculate  the 
number  of  hours  required  in  OJT  to  bring  an  airman  to  full 
proficiency.  These  figures  are  then  translated  into  OJT  labor 
costs . 

For  example,  an  airman  receives  training  for  a  particular  TM 
in  two  different  formal  training  settings,  a  technical  training 
center  and  a  field  training  detachment,  for  1  and  2  hrs, 
respectively.  Using  the  allocation  curve  (figure  3)  developed 
for  this  particular  TM,  we  find  that  1  hr  of  training  at  the 
technical  training  center  results  in  a  20%  proficiency  and  2  hrs 
of  training  at  an  FTD  results  in  a  40%  proficiency.  Thus,  the 
airman  is  60%  proficient  on  this  particular  TM  after  attending 
the  associated  formal  training  courses.  The  remaining  40% 
proficiency  (i.e.,  assuming  100%  proficiency)  is  attributed  to  1 
hr  of  training  in  an  OJT  setting.  OJT  costs  for  the  TM  are 
calculated  accordingly. 

Representative  Sites;  Travel  Costs  and  Training  Capacities. 
We  are  slowly  filling  in  our  picture/description  of  the  training 
system.  Thus  far,  we  have  identified  the  training  course  and  job 
assignment  flows  for  an  AF  specialty  (the  Aircraft  Environmental 
Systems  specialty  in  particular),  described  the  jobs  and  training 
courses  within  the  specialty  in  terms  of  TMs,  determined  the 
types  and  quantities  of  resources  required  for  training  each  TM 
in  a  given  setting,  and  determined  the  labor  and  non-labor  costs 
associated  with  training  each  TM  in  a  given  setting.  We  still 
need  two  more  pieces  of  information  before  our  picture  is 
complete  and  we  can  begin  dynamic  simulations  of  the  training 
systems.  Both  pieces  of  information,  travel/TDY  costs  and 
training  capacities,  are  based  on  the  development  of 
representative  sites. 

A  representative  site  is  a  model  of  a  hypothetical  AF  unit 
used  to  characterize  several  actual  similar  AF  units  (Rueter, 
Feldsott,  and  Vaughan,  1989;  Rueter  and  Feldsott,  1989). 
Representative  sites  are  based  on  mission,  major  command, 
resource  availability,  and  job  composition.  Each  actual  AF  unit 
within  a  specialty  is  associated  with  a  representative  site  so 
that  local  variations  in  travel  and  TDY  costs  for  a  particular 
U&T  pattern  can  be  accounted  for  in  the  TDS  model.  The  use  of 
representative  sites  also  reduces  the  number  of  comparisons 
needed  to  evaluate  training  capacities. 

Each  representative  site  has  a  certain  training  capacity 
based  on:  1)  the  allocation  of  training  time  to  specific  TMs  in  a 
given  setting;  2)  resource  requirements  for  training  specific  TMs 
in  a  given  setting;  3)  the  flow  of  airmen  through  the  site  (or 
training  volume);  4)  the  combinations  of  TMs  which  must  be 
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trained  (i.e.,  job  and  course  content)  at  the  site;  and  5)  the 
resources  available  to  support  that  training  (Rueter  and 
Feldsott,  1989).  All  of  these  inputs  derive  from  previously 
formed  data  bases  except  the  last  one.  Data  regarding  resource 
availability  at  the  representative  sites  must  be  collected  to 
round  out  our  picture.  Based  on  all  of  these  inputs,  TDS 
estimates  training  capacities  for  a  given  U&T  pattern  and 
identifies  any  resource  limitations  which  might  exist  in  the 
training  system.  Thus,  while  it  might  be  cost  effective  to 
implement  a  certain  training  program,  the  training  system  may  not 
have  the  required  training  capacities. 

Training  System  Simulations.  TDS  analyses  are  dependent 
upon  thorough,  accurate  data  collection  procedures.  The  data 
bases  formed  by  these  procedures  provide  TDS  with  a  complete 
picture  of  the  AFS  being  examined.  Based  on  this  picture,  TDS 
simulates  the  flow  of  airmen  through  the  U&T  pattern  and  tallies 
the  costs  associated  with  their  training.  Each  simulated  airman 
or  "entity"  passing  through  the  U&T  pattern  develops  a  unique 
record  which  shows  their  own  training  history  in  terms  of  TMs . 
These  histories  allow  the  TDS  model  to  determine  the  training 
requirements,  formal  and  OJT,  for  each  airman  at  each  phase  of 
their  career  and  the  costs  incurred  because  of  such  training. 
Training  costs  vary  according  to  which  TMs  are  trained  and  in 
which  settings. 

Histories  differ  from  entity  to  entity  because  of  the 
probability-based  nature  of  the  TDS  model.  At  each  phase  in  the 
U&T  pattern,  the  likelihood  of  any  airman  receiving  a  particular 
training  course  or  particular  job  is  based  on  predetermined 
probabilities.  Thus,  for  example,  after  completing  Basic 
Military  Training,  an  airman  in  the  Aircraft  Environmental 
Systems  specialty  will  either  attend  the  basic  resident  course 
for  that  specialty  or  be  assigned  directly  to  a  job  (see  figure 
2).  The  TDS  analyst  determines  the  probability  of  either 
occurrence  based  on  an  analysis  of  the  actual  U&T  pattern.  A 
data  base  of  transition  probabilities  is  developed  for  the  entire 
U&T  pattern  according  to  the  TDS  analyst's  preliminary  evaluation 
of  the  training  course  and  job  assignment  flows. 

The  simulation  is  perhaps  best  understood  by  following  a 
single  entity  through  the  model  of  the  423X1  U&T  pattern  (Figure 
2),  although  the  actual  TDS  simulates  the  flow  of  hundreds  or 
thousands  of  entities  through  the  model  in  order  to  achieve  a 
"steady  state."  In  this  example,  our  simulated  airman  (entity) 
enters  Basic  Military  Training  with  a  specified  number  of  other 
airmen.  This  initial  training  is  mandatory.  Following  BMT,  our 
simulated  airman  will  either  attend  the  basic  resident  course  or 
bypass  it.  Due  to  random  selection,  our  simulated  airman  is  sent 
to  the  basic  resident  course,  and  his  training  history  is  updated 
to  show  the  training  he  received  in  that  course. 

Prior  to  his  first  assignment  or  job  phase,  the  TDS 
temporarily  groups  our  airman  together  with  all  of  the  other 
simulated  airmen  —  both  those  who  have  attended  the  resident 
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course  and  those  who  have  bypassed  it.  The  airmen  are  then 
distributed  from  this  common  collect  to  the  their  first 
assignment  jobs  according  to  predetermined  probabilities  and 
regardless  of  the  entity's  previous  background.  Our  simulated 
airman  is  assigned  to  the  Environmental  Systems  Maintenance  job 
—  the  largest  first  assignment  job  —  by  chance.  While  in  that 
job,  our  airman  has  the  opportunity  to  receive  additional 
training  courses  —  once  again  based  on  probability.  Some 
courses  are  related  to  the  job  itself  and  others  are  related  to 
the  airman's  time  in  service. 

Training  which  is  required  by  the  airman's  job  and  not 
received  from  formal  courses  must  be  trained  in  OJT.  Training 
histories  are  used  to  determine  OJT  requirements  for  each  airman. 
That  is,  the  TM  content  of  the  job  is  compared  to  the  TMs  in  the 
airman's  training  history  and  proficiency  deficits  are 
identified.  These  deficits  are  assumed  to  be  trained  in  OJT. 

Following  the  first  assignment,  TDS  gathers  all  of  the 
simulated  airmen  into  a  common  "collect”  before  distributing  them 
to  their  second  jobs.  The  collect  includes  airmen  who  have 
finished  their  first  assignment  in  the  423X1  specialty  and  airmen 
who  have  entered  from  other  specialties.  Once  again,  the 
simulated  airmen  are  distributed  according  to  predetermined 
probabilities,  regardless  of  their  prior  histories.  The  airmen 
enter  their  new  jobs,  receive  the  associated  training,  and  are 
collected  into  a  common  pool  before  being  assigned  to  their  next 
jobs.  This  pattern  is  repeated  through  out  the  U&T  pattern. 

Costs  are  tabulated  according  to  the  training  each  entity 
receives . 

Alternative  Simulations;  Training  Decision  Impacts.  The 
cost  and  capacity  implications  of  training  decisions  can  be 
evaluated  by  modifying  the  model  of  the  current  U&T  pattern. 
Changes  are  made  to  the  appropriate  data  bases  within  the  model 
in  order  to  simulate  the  particular  training  decision  or  policy 
being  examined.  At  the  beginning  of  the  paper,  we  wanted  to 
examine  the  impact  of  a  reduction  in  TDY-to-school  funding  on  the 
Aircraft  Environmental  Systems  specialty.  After  developing  a 
model  of  the  current  U&T  pattern  for  that  specialty,  simulating 
the  impact  of  a  reduction  in  TDY-to-school  funding  becomes  a 
matter  of  changing  data  bases.  The  results  in  Table  1  represent 
a  reduction  in  TDY-to-school  funding  for  the  423X1  specialty  as 
it  was  actually  modeled  in  TDS.  This  reduction  in  funding  was 
modeled  by  reducing  the  probability  (i.e.,  in  the  appropriate 
data  base)  that  airmen  would  receive  training  in  courses  which 
required  'TDY  travel . 

The  cost  implications  for  OJT,  as  well  as  for  the  training 
system  in  general,  can  be  seen  in  Table  1.  Reducing  the 
probability  that  an  airman  will  attend  off-site  training 
increases  the  training  burden  and  cost  of  OJT  while  reducing  the 
costs  associated  with  formal  training  courses.  An  overall 
savings  is  realized  by  reducing  the  probability  of  TDY-to-school. 
The  values  shown  in  table  1  do  not  represent  actual  dollars, 
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rather  they  represent  relative  dollar  amounts. 

Other  training  decisions  can  also  be  modeled  by 
restructuring  the  U&T  pattern  and/or  changing  the  appropriate 
data  bases.  It  is  up  to  the  TDS  user  to  determine  exactly  how  a 
particular  training  decision  will  be  represented.  The  user,  for 
example,  may  choose  to  represent  a  reduction  in  TDY-to-school 
funding  by  eliminating  training  courses  from  the  model  rather 
than  reducing  the  probability  of  attendance  at  the  courses. 


TRAINEE/ 

TRAINER 

COSTS 

TDY 

COSTS 

OJT 

COSTS 

TOTAL  AFS 
TRNQ  COSTS 
PER  YEAR 

CURRENT  U&T 

1,081,000 

1.010,000 

556,000 

2,647,000  ' 

REDUCED 

TRAVEL 

952,000 

i 

864,000 

562,000 

2,378,000 

DIFFERENCE 

-129,000 

[  -146.000 

*6,000 

i 

-269,000  ! 

1 

PERCENT 

-11.93% 

-14.46% 

*1.06 

-10.16%  i 

DIFFERENCE 

! 

i 

j 

MTK.  MOOflCO  AS  aS»  MDUCTIOM  IN  TH|  FNOSABILITY  OF  ATTINIMNa  FTO,  PME, 

oirXbimNcto  couRSi 

Table  1 .  TDS  Analysis  of  the  impact  of  reduced  TDY-to-school 
funding  on  OJT  requirements  within  AFS  423X1 

Supplemental  Example .  Examining  the  impact  of  reduced  TDY- 
to-schoc 1  funding  on  the  Aircraft  Environmental  Systems  specialty 
represents  one  specific  application  or  use  of  the  TDS.  Such  an 
analysis  was  brought  about  by  budgetary  concerns,  i.e.,  the  lack 
of  funding.  The  Training  Decisions  System  can  be  used  to  analyze 
the  cost  and  capacity  implications  of  a  wide  variety  of  issues 
and  not  simply  those  driven  by  budgetary  concerns.  For  exeunple, 
a  training  manager  may  use  TDS  to  resolve  a  training  capacity 
problem  resulting  from  limited  resources  in  one  or  more  training 
settings;  determine  the  impact  of  a  personnel  policy  which  allows 
maintenance  specialists  to  move  between  large  and  small  aircraft; 
or,  exaunine  the  implications  of  shorter  terms  of  enlistment 
(i.e.,  which  will  reduce  the  average  experience  level  in  a  given 
specialty) . 

Following  is  an  example  of  a  training-related  decision  which 
can  be  modeled  in  TDS.  In  this  example,  training  managers  wish 
to  examine  the  cost  and  resource  capacity  implications  of 
removing  the  initial  technical  course  for  the  Aircraft 
Environmental  Systems  specialty.  Such  a  move  would  obviously 
reduce  the  cost  incurred  by  Air  Training  Command  (ATC)  who 
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provides  the  initial  technical  course,  but  the  burden  for 
training  those  tasks  taught  in  the  initial  technical  course  would 
shift  to  the  operational  units.  (ATC  could  arrange  to  train 
these  tasks  at  dispersed  locations,  such  as  FTD,  and  thus  avoid 
shifting  the  training  burden  to  operational  units.  For  this 
example,  however,  let  us  assume  that  ATC  has  not  planned  for  such 
provisions . ) 

The  TDS  analyst  would  have  to  interpret  the  problem 
described  above  in  terms  of  the  TDS  model  for  the  Aircraft 
Environmental  System  specialty  (Figure  2)  and  decide  which  data 
bases  to  change.  The  simplest  way  to  simulate  the  elimination  of 
the  initial  technical  school  course  is  to  remove  that  course  (and 
its  training  content)  from  the  U&T  pattern.  The  TDS  model  would 
then  simulate  the  flow  of  personnel  from  Basic  Military  Training 
(BMT)  straight  into  one  of  the  jobs  available  in  during  a 
specialist's  first  assignment.  Distribution  of  airmen  to  these 
jobs  would  be  based  on  prior  probabilities.  The  results  of  this 
analysis  are  shown  in  Table  2. 


ENTRY  LEVEL 
COURSE  , 

OJT  COST 

I 

OJT  HOURS  ; 
OJT  CAPACITY 


NO 

ENTRY-LEVEL 

CURRENT  COURSE  DIFFERENCE 


$941,517  $0 


530,189 

55.662  HRS 
NOT 

EXCEEDED 


808.417 
85,038  HRS 
EXCEEDED 


$-941,517 

278,228 
29,416  HRS 


1 

J 


$663,289j 


Table  2.  TDS  analysis  on  the  impact  of  eliminating  the 
initial  technical  school  course  for  AFS  423X1 

Eliminating  the  initial  technical  school  course  for  the 
Aircraft  Environmental  Systems  specialty  appears  to  save  dollars 
for  the  overall  training  system.  This  would  imply  that  it  is 
more  cost-effective  to  train  the  tasks  covered  by  the  technical 
school  course  on-the-job  rather  than  at  the  technical  school. 
However,  the  analysis  also  shows  that  the  operational  units  do 
not  have  adequate  resources  to  support  the  training  associated 
with  these  tasks.  The  resource  requirements  exceed  their 
capacity.  Consequently,  resource  limitations  must  be  resolved 
prior  before  elimination  of  the  initial  technical  school  course 
is  feasible.  The  alternative  is  to  eliminate  the  technical 
school  course  and  accept  decrease  in  proficiency  levels  and  the 
associated  consequences. 
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Sununary 


TDS  develops  cost  and  capacity  information  for  a  specialty 
by  simulating  the  flow  of  airmen  through  the  training  course  and 
job  assignment  flows  (U&T  pattern)  which  characterize  it.  The 
TDS  user  can  simulate  a  number  of  different  training  decisions  by 
manipulating  the  data  bases  and  structure  of  a  specialty's  U&T 
pattern.  Developing  a  model  of  the  U&T  pattern  itself  requires  a 
substantial  data  collection  effort:  jobs  and  training  courses 
must  be  identified  and  described  in  terms  of  their  task  content; 
training  course  and  job  assignment  flows  must  be  identified;  and, 
cost  and  resource  information  must  be  collected  and  analyzed.  In 
the  end,  a  thorough  data  collection  effort  provides  the  user  with 
a  versatile  and  robust  model  of  a  specialty's  training  program. 

The  purpose  of  this  paper  was  to  give  an  overview  of  the 
emerging  Training  Decisions  Modeling  Technologies.  Training 
planners  often  make  training  decisions  without  knowing  the  cost 
and  capacity  implications  of  their  decisions.  The  Training 
Decision  Modeling  Technologies  under  development  by  the  US  Air 
Force  offer  a  potential  solution  to  this  problem.  The  Training 
Decisions  System,  which  forms  the  baseline  technologies  in  the 
TDMT  research  program,  provides  training  planners  with  tools  and 
methodologies  for  examining  training  decisions. 
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